Quality control of automated whole-slide analyses

ABSTRACT

The subject disclosure presents systems and methods for automatically selecting meaningful regions on a whole-slide image and performing quality control on the resulting collection of FOVs. Density maps may be generated quantifying the local density of detection results. The heat maps as well as combinations of maps (such as a local sum, ratio, etc.) may be provided as input into an automated FOV selection operation. The selection operation may select regions of each heat map that represent extreme and average representative regions, based on one or more rules. One or more rules may be defined in order to generate the list of candidate FOVs. The rules may generally be formulated such that FOVs chosen for quality control are the ones that require the most scrutiny and will benefit the most from an assessment by an expert observer.

CROSS-REFERENCE TO RELATED APPLICATIONS

This patent application is a continuation of U.S. application Ser. No.15/659,654 filed on Jul. 26, 2017, which is a continuation ofInternational Patent Application No. PCT/EP2016/051865 filed Jan. 29,2016, which claims priority to and the benefit of U.S. ProvisionalApplication No. 62/110,472 filed Jan. 31, 2015. Each of the above patentapplications are hereby incorporated by reference herein in theirentireties.

BACKGROUND OF THE SUBJECT DISCLOSURE Field of the Subject Disclosure

The present subject disclosure relates to imaging for medical diagnosis.More particularly, the present subject disclosure relates to a methodfor analyzing a stained biopsy or surgical tissue sample beingimplemented by an image processing system and quality control ofautomated whole-slide analysis of tissue specimens.

Background of the Subject Disclosure

In the field of digital pathology, biological specimens such as tissuesections, blood, cell cultures and the like may be stained with one ormore stains and analyzed by viewing or imaging the stained specimen.Observing the stained specimen, in combination with additional clinicalinformation, enables a variety of processes, including diagnosis ofdisease, prognostic and/or predictive assessment of response totreatment, and assists in development of new drugs to fight disease. Asused herein, a “target” or “target object” is a feature of the specimenthat a stain identifies. A target or target object may be a protein,protein fragment, nucleic acid, or other object of interest recognizedby an antibody, a molecular probe, or a non-specific stain. Thosetargets that are specifically recognized may be referred to asbiomarkers in this subject disclosure. Some stains do not specificallytarget a biomarker (e.g. the often used counterstain hematoxylin). Whilehematoxylin has a fixed relationship to its target, most biomarkers canbe identified with a user's choice of a stain. That is, a particularbiomarker may be visualized using a variety of stains depending on theparticular needs of the assay. Subsequent to staining, the assay may beimaged for further analysis of the contents of the tissue specimen. Animage of an entire slide is typically referred to as a whole-slideimage, or simply whole-slide.

Quantitative analysis of a whole-slide, such as counting target objectssuch as cells of a certain kind, or the quantitation of a stainingresponse for all cells on a slide, is not feasible for human observers.Typically, a whole-slide contains several thousand to several hundredthousand cells, of which all or just a fraction may be relevant for ananalysis question at hand. Methods from image analysis, computer vision,and pattern recognition can be used for an automated quantitativeanalysis. Such automation with computer software enables quantitativewhole-slide analyses.

Current implementations of whole-slide analyses allow the user to selecta number of FOVs for quality control (QC). These FOVs are randomlyselected or systematically sampled by the software from all FOVs in theanalyzed tissue. A system for automated whole-slide analysis may presenta low-magnification view of the whole tissue that shows the position ofFOVs for QC. For example, FIG. 1A depicts a low-magnification view, withred rectangles 101 depicting positions of FOVs. For each of thesepositions 101, an FOV image of higher magnification may be generated asshown in FIG. 1B. A disadvantage of this method is that not all thetissue is presented for QC, only the selected FOVs. Errors of thealgorithm in tissue regions that are not visible in the presented FOVscannot be detected by the observer. Even if QC images and FOV selectionsare interactively generated, for instance by presenting a lowmagnification image such as the one in FIG. 1A in a graphical userinterface (GUI) enabling a user to select (for example with a mouseclick) a region of interest, this process is also tedious and requiresthe observer to interact with the GUI until she or he is satisfied thatall relevant regions of tissue have been observed.

SUMMARY OF THE SUBJECT DISCLOSURE

Embodiments of the invention provide a method for analyzing a stainedbiopsy or surgical tissue sample and a respective image processingsystem as claimed in the independent claims. Further embodiments of theinvention are given in the dependent claims.

In accordance with embodiments of the invention a whole-slide image isreceived by the image processing system. For example, the whole-slideimage that has been previously obtained from a histopathological tissueslide of a stained tissue sample is read from an electronic memory ofthe image processing system for processing in order to obtain a resultof the detection of one or more biological target objects. Hence, byreading the whole-slide image from electronic memory, the whole-slideimage is received. Embodiments of the invention may also encompass theacquisition of the whole-slide image by means of an optical sensor, suchas a microscope. In the latter case the whole-slide image may bereceived by performing an image acquisition step using the sensor.

The whole-slide image is analyzed by execution of an image processingalgorithm for detection of biological target objects. Depending on theimplementation, a quantitative result is obtained, such as a count for agiven target object that indicates the overall number of target objectswithin the whole-slide image or within a region of the whole-slideimage. The quantitative result may be output on a display of the imageprocessing system and visualized as a heat map.

The variation of the intensity and appearance of a staining of tissue onthe histopathological tissue slide, staining artefacts and otherhistopathological artefacts may cause an erroneous detection of a targetobject by execution of the image processing algorithm. Suchhistopathological artefacts are as such known, cf. JOMPF REVIEW ARTICLE,Year: 2014, Volume: 18, Issue: 4, Page: 111-116 “Artefacts inhistopathology” Shailja Chatterjee(http://www.jomfp.in/printarticle.asp?issn=0973-029X; year=2014;volume=18; issue=4; spage=111; epage=116; aulast=Chatterjee) whichprovides a comprehensive overview of histopathological artefacts, suchas forceps artefacts, crush artefacts, fulguration artefacts, fixationartefacts, artefacts introduced during specimen transport, freezingartefacts, processing artefacts, artefacts due to folds and others.

Depending on the intensity and appearance of a staining of tissue on theslide or the presence of artefacts, the computer software, i.e. theimage processing program that implements the image processing algorithm,might miss target objects, such as cells, that should be included in theanalysis or misinterpret non-target structures (for example, stainingartefacts) as target objects.

In accordance with embodiments of the invention, quality control imagesshow the analyzed tissue together with algorithm detection results, forexample as color-coded marks and symbols overlaid on the original image.In order to see these algorithm results on a cell-by-cell level, such animage my be shown in magnification, such that only a small fraction ofall tissue can be presented in one image.

In accordance with embodiments of the invention a quality check isperformed on the whole-slide image using the target objects that havebeen potentially erroneously detected by execution of the imageprocessing algorithm.

For performance of the quality check FOVs of the whole-slide image aredefined and at least one rule is applied to the FOVs for checkingwhether a criterion is fulfilled for a given FOV. If the criterion isfulfilled, this indicates a risk for that FOV containing ahistopathological artefact and/or erroneous analysis results caused byimperfections of the image processing algorithm, i.e. an erroneousdetection result as regards the detection of one or more target objectsin the given FOV that may have been caused not by a histopathologicalartefact but by the image processing algorithm itself.

For example, the total number of biological target objects detected in agiven FOV by the image processing algorithm is calculated and the FOVsare sorted by the number of the respective total target objects inaccordance with the rule. If the total number of target objects within aFOV is either very high or very low this may indicate the presence of ahistopathological artefact and/or erroneous analysis results. In thiscase the criterion that is associated with the rule may be that a givenFOV is a top ranking or a low ranking FOV on the sorted list. Inparticular, the topmost and the bottom FOV from the list are selected.

At least a sub-set of the set of FOVs that is identified by applying theat least one rule is displayed on a display of the image processingsystem for review by an observer, such as a pathologist. The imageprocessing system may comprise a graphical user interface that enablesthe image processing system to receive an entry signal from the observerthat indicates whether the quality of an image portion of one of thedisplayed FOVs from the sub-set is sufficient for the analysis. In otherwords, if the observer identifies a histopathological artefact in adisplayed FOV of the sub-set, he or she may operate an element of thegraphical user interface, such as by a mouse click, pressing a button orusing a data entry stylus, to signal to the image processing system thatthe respective FOV contains a histopathological artefact.

In the event that such an entry signal is received by the imageprocessing system, the result of the detection of the biological targetobjects that is output by the image processing system may be an errorsignal for signaling that automated analysis results in the displayedtissue region are not correct or not meaningful, for example because thequality of the whole-slide image is insufficient for performing ananalysis.

Alternatively, a portion of the whole-slide image is selected, such asby means of an annotation tool, and excluded from the analysis. In thelatter case, the result that is output by the image processing systemindicates the detected biological target objects that have been obtainedby execution of an image processing algorithm but excluding targetobjects that have been identified in the excluded image region thatcontains the histopathological artefact or erroneous analysis results.The latter case has the advantage, that the stained biopsy tissue samplemay still be used for research and/or diagnostic purposes with therequired degree of scientific certainty due to the exclusion of theimage area that contains the histopathological artefact withoutburdening the observer with a lengthy and tedious review process of alarge number of FOVs as due to the application of the at least one ruleonly candidate FOVs are displayed that are likely to contain anartefact.

Embodiments of the present invention are particularly advantageous as aquality control is introduced into an image processing system thatrelies on the image processing algorithm that is used for the detectionof biological target objects; no specific image processing algorithm forartefact detection is required.

For example, depending on the intensity and appearance of a staining oftissue on the slide or the presence of artefacts, the computer software,i.e. the image processing program that implements the image processingalgorithm, might miss target objects, such as cells, that should beincluded in the analysis or misinterpret non-target structures (forexample, staining artefacts) as target cells. In accordance withembodiments of the invention, quality control images show the analyzedtissue together with algorithm detection results, for example ascolor-coded marks and symbols overlaid on the original image. In orderto see these algorithm results on a cell-by-cell level, such an image mybe shown in magnification, such that only a small fraction of all tissuecan be presented in one image.

Embodiments of the invention are particularly advantageous as a highdegree of reliability of the quality control may be obtained by havingthe observer review only a minimal number of FOVs from the large numberof FOVs, such as hundreds, thousands or even millions FOVs. This isespecially important for a whole-slide image that covers a relativelylarge area and thus may contain a very large number of FOVs.

The subject disclosure solves the above-identified problems by providingsystems and methods for automatically selecting meaningful regions on awhole-slide image and performing QC on the resulting collection of FOVs.The automatic selection is performed subsequent to determining apresence and quantity of relevant objects such as cells based on aresponse to a staining assay applied to the tissue on the slide.Subsequent to the detection, density maps or heat maps may be generated,quantifying the local density of the detection results. The heat mapsmay depict positive and negative detection results, as well as objectsclassified as “object of no interest”, as well as a local ratio of celldensities. The heat maps as well as combinations of maps (such as alocal sum, ratio, etc.) may be provided as input into an automated FOVselection operation. The selection operation may select regions of eachheat map that represent extreme and average representative regions,based on one or more rules. One or more rules may be defined in order togenerate the list of candidate FOVs. The rules may generally beformulated such that FOVs chosen for quality control are the ones thatrequire the most scrutiny and will benefit the most from an assessmentby an expert observer. The disclosed operations therefore mitigatetedious whole-slide QC procedures that are either interactive or requirea large number of FOVs, while also avoiding incomplete QC whereartefacts can be missed by choosing FOVs that cover the extreme cases ofdetected target and non-target objects.

In one exemplary embodiment, the subject disclosure provides a systemfor quality control of automated whole-slide analysis, including aprocessor, and a memory coupled to the processor, the memory to storecomputer-readable instructions that, when executed by the processor,cause the processor to perform operations including applying a rule to aplurality of fields-of-view (FOVs) on a whole-slide image of a tissuespecimen, and presenting a set of FOVs that match the rule to anobserver for a quality control analysis.

In another exemplary embodiment, the subject disclosure provides acomputer-implemented method for quality control of an automatedwhole-slide analysis, including determining a local density of objectsof one or more types in a whole-slide image, selecting a plurality ofrules based on the local density, and selecting a set of fields-of-view(FOVs) that fulfill the plurality of rules, wherein one of the rulessets a maximum number of FOVs within the set of FOVs.

In yet another exemplary embodiment, the subject disclosure provides atangible non-transitory computer-readable medium to storecomputer-readable code that is executed by a processor to performoperations including segmenting a whole-slide image into a plurality offields-of-view (FOVs), and selecting a set of FOVs that match one ormore rules for the selection of FOVs, wherein the rules are applied toheat maps representing a local density of one or more objects ofinterest in the whole-slide image, and wherein the set of FOVs arepresented for quality-control of an image analysis algorithm applied tothe whole-slide image

BRIEF DESCRIPTION OF THE DRAWINGS

FIGS. 1A-B depict positions of FOVs determined using prior art methods.

FIG. 2 depicts a system for quality control of automated whole-slideanalysis, according to an exemplary embodiment of the present subjectdisclosure.

FIGS. 3A-3B depict a method for quality control of automated whole-slideanalysis, according to an exemplary embodiment of the present subjectdisclosure.

FIG. 4 depicts heat maps showing the density of detection results fortissue for a whole-slide image, according to an exemplary embodiment ofthe subject disclosure.

FIGS. 5A-B depict initial sets of candidate FOVs, according to anexemplary embodiment of the subject disclosure.

FIGS. 6A-6B depict examples of FOVs that are automatically selected forquality control (QC) via the disclosed operations, according to anexemplary embodiment of the subject disclosure.

FIG. 7 is illustrative of an embodiment of a method of the invention.

DETAILED DESCRIPTION OF THE SUBJECT DISCLOSURE

The subject disclosure solves the above-identified problems by providingsystems and methods for automatically selecting meaningful regions on awhole-slide image or portion thereof of a specimen such as a biologicalspecimen, and performing QC on the resulting collection of FOVs. Theautomatic selection is performed subsequent to determining a presenceand quantity of relevant objects such as cells based on a response to astaining assay applied to the specimen, for example a tissue specimen onthe slide. Subsequent to the detection, density maps or “heat maps” maybe generated. The heat maps quantify the local density of the detectionresults. The heat maps may, for instance, depict positive and negativedetection results, as well as objects classified as “object of nointerest”, as well as a local ratio of cell densities, such as a ratioof all cells positive for biomarker 1 that are also positive forbiomarker 2. Separate heat maps may be generated for each biomarker ordetection result, as depicted in FIGS. 4A-4F. The heat maps as well ascombinations of maps (such as a local sum, ratio, etc.) may be providedas input into an automated FOV selection operation. The selectionoperation may select regions of each heat map that represent oppositeextremes and average representative regions, based on one or more rules.One or more selection rules may be defined in order to select a sub-setof FOVs for QC out of a larger list of candidate FOVs. The rules maygenerally be formulated such that FOVs chosen for quality control arethe ones that require the most scrutiny and will benefit the most froman assessment by an expert observer. For example, the selected FOVs maybe considered candidates for quality control if they contain regionswith the highest, an average, or the lowest values on a heat map or on acombination of heat maps.

FIG. 2 depicts a system 200 for quality control of automated whole-slideanalysis, according to an exemplary embodiment of the subjectdisclosure. System 200 comprises a memory 210, which stores a pluralityof processing modules or logical instructions that are executed byprocessor 205 coupled to computer 201. An input 202 may trigger theexecution of one or more of the plurality of processing modules. Input202 includes user inputs as well as inputs supplied over a network to anetwork server or database for storage and later retrieval by computer201. Besides processor 205 and memory 210, computer 201 also includesuser input and output devices such as a keyboard, mouse, stylus, and adisplay/touchscreen. As will be explained in the following discussion,processor 205 executes logical instructions stored on memory 210,performing image analysis to detect and segment objects using imageanalysis module 211, quantify and depict results using heat mapsgeneration module 112, select FOVs for QC using FOV selection module213, and store and retrieve rules for heat map generation and FOVselection in rules database 214.

The input 202 may include information about a target tissue type orobject, as well as an identification of a staining and/or imagingplatform. For instance, the sample may have been stained by means ofapplication of a staining assay containing one or more differentbiomarkers associated with chromogenic stains for brightfield imaging orfluorophores for fluorescence imaging. Staining assays can usechromogenic stains for brightfield imaging, organic fluorophores,quantum dots, or organic fluorophores together with quantum dots forfluorescence imaging, or any other combination of stains, biomarkers,and viewing or imaging devices. Moreover, a typical sample is processedin an automated staining/assay platform that applies a staining assay tothe sample, resulting in a stained sample. There are a variety ofcommercial products on the market suitable for use as the staining/assayplatform, one example being the Discovery™ product of the assigneeVentana Medical Systems, Inc. Input information may further includewhich and how many specific antibody molecules bind to certain bindingsites or targets on the tissue, such as a tumor marker or a biomarker ofspecific immune cells. Additional information input into system 100 mayinclude any information related to the staining platform, including aconcentration of chemicals used in staining, a reaction times forchemicals applied to the tissue in staining, and/or pre-analyticconditions of the tissue, such as a tissue age, a fixation method, aduration, how the sample was embedded, cut, etc. Input 202 may beprovided via an imaging system, for example a camera on a microscope ora whole-slide scanner having a microscope and/or imaging components, ormay be provided via a network, or via a user operating computer 201.

Image analysis module 211 may be executed in order to detect and/orsegment objects of interest within the received image. The objects maybe detected, i.e. the presence and location of the objects is reportedby the processing, or segmented, i.e. one or more objects are detectedand delineated by assigning a plurality of pixels on the wholeslideimage to one of either the one or more objects or a background. Severaldifferent objects may be detected or segmented and may have differentbiological meanings that are distinguished to answer a biologicalquestion at hand. One possible biological meaning is “object of nointerest”, indicating that a detection or segmentation result representsnon-target staining, slide artefacts, staining too faint to be ofrelevance, etc. For example, while scoring the biomarker Ki-67, alltumor nuclei are counted and classified into either positively ornegatively stained. “Objects of no interests” in this case are thenuclei of all cells on the slide that are not tumor cells, such asimmune cells or stroma cells. More generally, the results of imageanalysis module 211 may be a list of positions in the image whereobjects have been detected and/or segmented. Optionally, the resultsfrom module 211 further include a category or class that furtheridentifies each detected object.

Heat map generation module 212 may be executed to quantify the resultsof detection and segmentation operations performed by image analysismodule 211. A heat map maps the density of various structures on thewhole-slide image. Heat maps may be generated depending on thebiological question at hand (which also dictates the type of imageanalysis performed). For example, heat maps may depict positive andnegative detection results for objects such as cells, nuclei, etc. Heatmaps may further depict “object of no interest”, as well as a localratio of cell densities, such as a ratio of density of one type of cellversus another, or densities of co-location of biomarkers. Separate heatmaps may be generated for each biomarker or detection result, asdepicted in FIGS. 4A-4F.

FIG. 4, for example, depicts heat maps showing the density of detectionresults for tissue for a whole-slide image (a). The local density forcells positive only for biomarker 1 is shown in (b), local density forbiomarker 2 is shown in (c), positive density for both biomarkers isshown in (d), density of the detection of “objects of no interest” isshown in (e), and the local ratio of cells positive for biomarker 2 tocells positive for biomarker 1 is shown in (f). Referring back to FIG.2, heat maps may be generated based on heat map rules stored in ruledatabase 214, or rules input via input 202. Exemplary heat map rules arefurther described herein. Further, database 214 may also include staininformation relating to various combinations of stains andcounterstains, enabling automated generation of rules, or to mitigateconflicts in rule generation or rule input.

The heat maps and their associated data (such as combinations ofquantities depicted in each heat map) may be provided as input into FOVselection module 213. FOV selection module 213 may select regions ofeach heat map based on selection rules stored in rule database 214 orrules input via input 202. For example, selection rules may require thatFOVs representing high and low extremes of detection result densitiesshould be selected and presented for QC. The selection rules may bedefined in order to select FOVs for QC from the list of candidate FOVsand may generally be formulated such that FOVs chosen for qualitycontrol are the ones that require the most scrutiny and will benefit themost from an assessment by an expert observer. For example, the selectedFOVs may be considered candidates for quality control if they containregions with the highest, an average, or the lowest values on a heat mapor on a combination of heat maps. Operations performed by FOV selectionmodule 213 are further described with reference to FIGS. 3A-3B.

As described above, the modules include logic that is executed byprocessor 105. “Logic”, as used herein and throughout this disclosure,refers to any information having the form of instruction signals and/ordata that may be applied to affect the operation of a processor.Software is one example of such logic. Examples of processors arecomputer processors (processing units), microprocessors, digital signalprocessors, controllers and microcontrollers, etc. Logic may be formedfrom signals stored on a computer-readable medium such as memory 210that, in an exemplary embodiment, may be a random access memory (RAM),read-only memories (ROM), erasable/electrically erasable programmableread-only memories (EPROMS/EEPROMS), flash memories, etc. Logic may alsocomprise digital and/or analog hardware circuits, for example, hardwarecircuits comprising logical AND, OR, XOR, NAND, NOR, and other logicaloperations. Logic may be formed from combinations of software andhardware. On a network, logic may be programmed on a server, or acomplex of servers. A particular logic unit is not limited to a singlelogical location on the network. Moreover, the modules need not beexecuted in any specific order. Each module may call another module whenneeded to be executed.

FIGS. 3A-3B show a method for quality control of automated whole-slideanalysis, according to an exemplary embodiment of the subjectdisclosure. The operations described in this exemplary embodiment mayuse components described with reference to system 200, or othercomponents that perform similar functions. For example, referring toFIG. 3A, an input may be provided (S301) that includes a whole-slideimage or portion thereof, as well as results of an analysis operationthat may detect or segment objects on the slide, and provide one or morecategories or classes of objects as its results. For example, inputinformation may include:

The wholeslide image that was analyzed by an image analysis algorithmknown in the art,

The position of all objects detected by the automated algorithm, and

The object class for all objects as provided by the automated algorithm.

The image analysis results may include, for instance a list of objectsof type A and their positions in the wholeslide image together with alist of objects of type B and their position in the wholeslide image.Examples for objects that are algorithm results include the positions ofnegative nuclei and the position of positive nuclei, the position ofnuclei with no membrane staining, with weak membrane staining, and withstrong membrane staining, etc. Any such analysis algorithm can be inputand used to automatically determine FOVs to QC the results from such analgorithm. For example, image analysis operations may detect and countcells of two different types A and B that have been stained on thetissue specimen. It has been assumed that staining, detecting, andsegmentation of these cells are not perfect, such that weak staining,non-target staining or a misinterpretation of segmentation and detectionis possible. Assuming that the detection and segmentation operationsprovide such a list of results, the list may be classified into eitherbeing a cell of type A, a cell of type B, or not a target cell.Additional information input may include information about the tissueand target structures such as cells or other structures of interest,specific antibody molecules, a choice of biomarkers, and any informationrelated to the staining platform, as well as user defined rules forselection of FOVs, as further described herein. The image along withanalysis results and additional information may be input and parsed(S302) to determine optimal candidate FOV creation (S303) and selectionbased on rules (S304).

Upon parsing this input (S302), a plurality of candidate FOVs may becreated (S303). Typically, a FOV size is chosen such that a FOV can bepresented on a computer screen at full resolution. For example, if acomputer screen offers 1000×1000 pixels resolution and the pixel in thewhole-slide image is 0.5 micrometer×0.5 micrometer, then a good FOVcandidate size is also 1000×1000 pixels or 0.5 mm×0.5 mm in size.Generally, FOV candidates are created to cover all tissue in the image.This process may be referred to as “tiling”. FIGS. 5A-B depict initialsets of candidate FOVs, according to an exemplary embodiment of thesubject disclosure. FIG. 5A depicts candidate FOVs 520 created by tilingof all relevant tissue without any overlap. It may be observed thatoptimal FOV candidates are not likely to coincide with such a tilinggrid. Therefore, candidates may be created at many more positions. FIG.5B depicts additional FOVs 521 added to FOVs 520 for an overlap ofapproximately 50%. In the color version of this drawing, alternate tilesare shown in red 520 and black 521, respectively. In some extreme cases,every pixel of relevant tissue may be considered as the center of a FOV,which would provide many thousand FOV candidates.

These initial lists of candidate FOVs are generated with FOVs of aspecific size, based on whether or not they may be viewed on a computermonitor, on a printed sheet of paper, etc. at sufficient magnificationto perform the quality control. The image may have been scanned prior toany analysis, and thus the magnification and pixel size is generallyfixed. Therefore, FOV size and resolution for quality control should fitonto a screen. Each screen can display a fixed number of pixels, andthat is the size of an FOV, unless a user requests a different size. Forexample, FOV sizes may be adjusted to match a user's preference. A userpreference might be “I prefer more smaller FOVs to view it on my smalllaptop” or “I can assess the algorithm results even at a lowermagnification.” The FOV candidates are typically created for regions inthe image that contain tissue, and that no FOV candidates are createdfor regions of the slide that contain no tissue.

Moreover, FOVs do not have to be squares or rectangles, althoughrectangles enable optimal display on a computer screen or sheet ofpaper. However, free-form FOV shapes may also be defined, such as roundFOVs, if the option is available based on the system implementation.Generally, FOVs are automatically created on each wholeslide image andpresented on a computer monitor, printed if necessary, or viewed in anyexisting digital pathology viewing application. For example, awholeslide viewer (such as the assignee VENTANA's VIRTUOSO viewer) maybe used to automatically navigate to a FOV selected for quality controlbased on the operations described herein, enabling a user to zoom andpan through the image. Such a viewer may automatically navigate to theFOVs selected out of the original list, with pan and zoom optionsprovided.

Referring back to FIG. 3A, FOVs may be selected and/or sorted (S304)based on application of one or more selection rules. Generally, apreference is to select a small number (for example, 5) of FOVs from the100 or more candidate FOVs tiled on the wholeslide image. The sub-set of5 FOVs may be the ones that meet criteria defined by selection rules.These rules may be defined in and retrieved from rules database 314, orany other source including but not limited to input information S301. Asdescribed herein, the rules may generally be formulated such that FOVschosen for quality control are the ones that require the most scrutinyand will benefit the most from an assessment by an expert observer. Forexample, one set of rules may include the following choices orpreferences:

An FOV with highest density of cells of type A,

an FOV with highest density of cells of type B,

an FOV with highest ratio of cells of type A (where a number of cells oftype A divided by a sum of number of cells of type A or B is highest),

an FOV with highest ratio of cells of type B,

an FOV with median density of cells of type A and B together, and

an FOV with the highest density of detection results that wereclassified as “not a target cell” (this may be selected from anintermediate output from an image analysis algorithm).

Application of the above set of rules to all FOVs would likely result ina choice of 6 FOVs for quality control that may be presented to anexpert observer via a display or other output device. However, otherrule sets may result in a number of target FOVs that is higher or lowerthan the number of rules. The rules are generally predefined, and a usermay be provided with an option to view and edit rules, if desired. Auser interface to modify rules may be in the form of a text orspreadsheet editor. For example, a spread sheet editor with pre-definedfields like “highest of”, “lowest of”, “nth percentile of” in the firstfield and “objects type A”, “objects type B, “objects type C”, “objectsof A and B”, etc. in the second field may be provided. Examples ofobject types include but are not limited to:

Tumor cells positive for Ki-67 and tumor cells negative for Ki-67.

Tumor cells positive for ER and tumor cells negative for ER

Tumor cells with no Her2 membrane staining, Tumor cells with weak Her2membrane staining, Tumor cells with intermediate Her2 membrane staining,and tumor cells with strong Her2 membrane staining,

Tumor cells, stroma cells, and immune cells.

T-Lymphocytes and B-Lymphocytes.

As described herein, the rules may be input by a pathologist, and may bepredefined such that set of rules that have been input and selected forresults from different slides may be applied to new input slides.Therefore, set of pre-defined rules may be provided that a pathologistcan choose instead of manually entering the rules.

Typically, there will be either more rules than desired FOVs, or moreFOVs desired than rules. For example, the rule that chooses “an FOV withmedian density of cells of type A and B together” is different from therules preferring a highest or lowest quantity. Many such rules may becreated that, in general, require the choice of “an FOV where thedensity of cells or any other quantitative result is close to a chosenpercentile from all FOVs.” To implement such a rule, the FOVs may haveto be sorted based on the requirement of the rule. For example, a rulemay request some percentile of “density of cells A and B together.” Sucha rule would result in a sorted list (where the number next to each FOVis a unique serial number or identifier of the listed candidate FOV):

FOV 13

FOV 84

FOV 22

FOV 93

( . . . more FOVs in the sorted list, not shown here . . . )

FOV 45

FOV 29

FOV 27

( . . . more FOVs in the sorted list, not shown here . . . )

FOV 15

FOV 73

The list is created such that the first listed FOV has the highestdensity. the second listed FOV has the second highest density, and so ondown to the last FOV, which has the lowest density. In such a list, FOVsmay be selected based on percentiles. The FOV with a median density (the50% percentile) is the FOV for which half of the FOVs have a higher celldensity, and the other half of the FOVs has a lower cell density.Similarly, an FOV at the 25% percentile is chosen such that one quarterof the other FOVs has a lower density, and three quarters have a higherdensity. An FOV at the 75% percentile is chosen such that three quartersof the other FOVs have a lower density, and only one quarter of theother FOVs has a higher density. In this example, FOVs 13 and 84 arerespectively highest and 2nd-highest, and FOVs 15 and 73 arerespectively 2nd-lowest and lowest. FOV 29 may be at the median, withhalf the FOVs higher in density and half the FOVs lower in density.

Given such a list, FOVs may also be sorted by “being close to the targetpercentile”. For each FOV, the absolute difference (for example in celldensity) is computed to the density of the FOV at the median, the 25%percentile, etc. An exemplary list of FOVs sorted by “difference indensity to 75th percentile” may provide the following result:

FOV 91

FOV 90

FOV 80

( . . . )

FOV 16

In this example, FOV 91 may be the FOV at the 75th percentile (i.e. zerodifference), FOV 90 may have the smallest difference in density, and FOV16 may be the FOV with the highest difference in density to that of the75th percentile (regardless of whether the density is higher or lower).Many other examples of rules and their applications may be contemplatedby those having ordinary skill in the art in light of this disclosure,and depending on the biological question that is presented.

When the list of FOVs meeting the criteria of the rules are sortedaccording to each of the chosen rules, the result may comprise as manysorted lists as there are rules, as depicted in the example of Table 1:

TABLE 1 Sorted by Rule 1 Sorted by Rule 2 . . . Sorted by Rule N FOV 13FOV 91 FOV 19 FOV 84 FOV 90 FOV 08 FOV 22 FOV 80 FOV 25 (. . .) (. . .)(. . .)

Referring back to FIG. 3A, for a wholeslide tissue image, the “Createcandidate FOV” operation will generate many FOV candidates, out of whichonly a small sub-set can be presented to a user for QC. As describedherein, the rules may either be pre-defined for each analysis algorithm,or may be selected by a user. These rules are used to select thissub-set. Two different operations “Select and purge” (S306) and“Optimization” (S308) determine that sub-set of FOVs out of allcandidate FOVs that best follows the rules. For example, a determinationis made if the number of requested FOVs for QC exceeds the number ofrules (S305). If there is a greater number of requested FOVs than numberof selection rules, then some FOVs may be selected for QC and purgedfrom the lists (S306). For example, each FOV that best matches a ruleand has the highest position in a list may be selected for QC. TheseFOVs may then be removed from that list and purged from all other listsin the table. For example, FOVs 13, 91, and 19 from Table 1 may bechosen for QC, and removed from all other tables, thereby reducing thetotal number of FOVs for the next round of selection and purging (S306)or optimization (S308). Another method for purging includes removing allFOVs that overlap with one of the chosen FOVs. This enables avoidingpresentation of the same tissue section in more than one FOV. Theselection and purging step (S306) may be repeated until the number ofFOVs that still have to be selected for QC to reach the number ofrequested FOVs is equal to or less than the number of selection rules.

If there are more rules than FOVs (S307), then an optimization (S308)may be necessary. For example, even after a set of FOVs has already beenselected as in the paragraph above, there may end up being fewer FOVsleft to obtain the number of requested FOVs than there are rules. Ineither case, it may be no longer possible to select the best FOV foreach rule. Many different sub-sets of FOVs can be selected from the listof candidate FOVs. The optimization process (S308) may now select FOVssuch that each rule is fulfilled as well as possible, for instance bycomputing a “quality” of the set of FOVs that are currently chosen fromthe tables. Finding the best sub-set may include applying known computerscience optimization problems. Therefore, many other formulas can beused as well.

FIG. 3B shows an exemplary optimization operation. One sub-set of FOVsmay be retrieved (S311) as an input into a quality determinationtogether with the tables where the sub-sets are sorted according todifferent criteria (rules). The result of this computation may comprisea number that indicates how good this sub-set is—it measures how “high”the FOVs appear in the table. For example the quality determination maycheck the position of each FOV in the sub-set (S312), and find the bestFOV in the list for the selection rule that is fulfilled worst (S313).Many other rules (average position of all FOVs in all lists in thetable, average position of the best FOV in each list in the table, etc.)can be used instead. This “quality” may be computed by checking theposition of the best FOV in each table. For example, if 3 FOVs arechosen for 4 rules, and the best FOV in each table is at the positions3, 5, 10, 2, respectively, than the quality of this set of FOVs is 10.The quality here is a measure that assesses how well a set of FOVsfulfills all the rules together. In the example given here, the positionof the best FOV in the list that is the least good fulfilled is used asmeasure for quality. A different set of FOVs might have the bestposition in each table at positions 6, 7, 7, 8, with the quality measure8. Another set of FOVs might have the best position in each table as 1,2, 1, 20 which results in a quality measure 20. For this qualitymeasure, the set with the lowest quality measure is considered the bestselection. As will be evident to those skilled in the relevant art,there are many methods to measure how well a set of FOVs fulfills a setof rules. For any of these quality measures, methods may be employed tooptimize the selection, i.e. to determine the set of FOVs that has thebest quality measure, with the simplest being the test of all possibleFOV selections and choosing the set with the best quality measure(S314). An exhaustive search over all possible FOV sub-sets may resultin a quality measure being computed for all possible FOV sub-sets, andthe sub-set of FOVs with the best quality measure being chosen (S314).As will be evident to those skilled in the relevant art, there are manymethods in addition to exhaustive search that can be used to optimizethe FOV sub-set to fulfill all rules. In an exemplary embodiment, apre-defined number of FOVs are chosen such that they together bestrepresent regions of highest, most representative, and lowest densitiesof detected objects. For example, one FOV can be chosen if it contains avery high density of a first cell type and a very low density of secondcell type, even if it does not contain the highest or lowest respectivedensity for these cell types. Multiple of these replacements andcompromises can be performed by the optimization step to cover allrelevant features on the slide in few FOVs. After selecting and purging(S306) FOVs and/or optimizing (S308) FOVs, the resultant set of chosenFOVs are output (S309) to an expert observer for analysis and QC.

FIGS. 6A-6B depict examples of FOVs that are automatically selected forquality control (QC) via the disclosed operations, according to anexemplary embodiment of the subject disclosure. FIG. 6A depicts theindividual FOVs 660, and FIG. 6B depicts the position of these FOVs 660in a whole-slide image. Referring to FIG. 6A, a region with the highestdensity of cells positive for only the first biomarker is depicted in(a), a region with the highest density of cells positive for only thesecond biomarker is depicted in (b), regions with highest density forboth biomarkers is depicted in (c), the highest density of “objects ofno interest” is depicted in (d), the highest ratio of cells positive forbiomarker 2 is depicted in (e), and the region with the lowest densityof stained cells of interest is depicted in (f). In this embodiment, theFOVs are selected based on rules that determine that these FOVs requireQC the most, for example, (b) could depict a strong non-target stainingfor biomarker 2 and stain 2 that could result in a false detection ofrespective cells, (c) may depict regions so strongly stained withbiomarker 1 and stain 1 that the analysis might misinterpret the cellsas being double-stained, (d) may depict a region with strong non-targetstaining caused by speckling from stain 1, and (e) may depict a regionwhere non target-tissue was analyzed. For the five maps presented inthis example case, choosing FOVs for the highest, an average, and thelowest value on each would result in 15 FOVs, and the application of theoptimization (S308) would provide these 5 FOVs to best fulfill such 15selection rules.

FIG. 7 is illustrative of a method for analyzing a stained biopsy tissuesample in accordance with an embodiment of the invention. A digitalwhole-slide image 700 that is obtained from a histopathological tissueslide of a stained biopsy tissue sample is received by the input 202 ofthe image processing system 200 (cf. embodiment of FIG. 2). The image700 is obtained in an image acquisition step 702 by means of an imagingsystem from a histopathological slide that carries a tissue sample, forexample by means of a camera on a microscope or a whole-slide scannerhaving a microscope and/or imaging components. The image 700 is storedin the electronic memory 210 of the image processing 200.

In step 704 the digital whole-slide image 700 is read from the memory210 for execution of an image processing program that implements animage processing algorithm for detection of biological target objects inthe whole-slide image. For example, the image processing algorithmserves for detecting the presence and/or shape of one or more types ofbiological target objects, such as cells, that may be stained by one ormore stains. Depending on the implementation a quantitative analysis isperformed by the image processing algorithm, such as a determination ofthe number of biological target objects that is present in thewhole-slide image or a portion of the whole-slide image 700.

For illustrative purposes FIG. 7 shows target objects 706, 708, 710, 712and 714 that have been detected by execution of the image processingalgorithm in step 704. It is to be noted that in a practicalimplementation a much larger number of target objects is typicallydetected.

The image processing program is executed by the processor 205 of theimage processing system 200. In step 716 FOVs are defined in thewhole-slide image 700 such as by invoking the FOV selection module 213.

In one implementation the FOVs are predefined by a grid 718, i.e. atiling grid as shown in FIG. 7. Alternatively predefined FOVs thatoverlap may be utilized or every pixel of relevant tissue is taken asthe center of a FOV which provides a very large number of FOVs. As afurther alternative a more complex automated algorithm may be utilizedfor defining the FOVs in step 716 such as a method known from PCT/EP2015/070100, international filing date 3 Sep. 2015, claiming priority toU.S. 62/045,484 the entirety of which being herein incorporated byreference.

In the example considered here, execution of step 716 results in anumber of 15 FOVs by the tiling grid 718.

In step 720 at least one of the rules stored in rule database 214 isread out from the electronic memory 210 by the processor 205 forapplication on the FOVs defined in step 716. Application of the at leastone rule in step 220 on the set of candidate FOVs results in a sub-setof FOVs that meet a predefined selection criterion. In the exampleconsidered here the sub-set contains the FOVs 722, 724 and 726.

In the following step 728 the FOVs that form the sub-set are displayedon a display of the computer 201 of image processing system 200, such ason a display 730.

In one implementation, the FOVs that form the sub-set are displayedsequentially one after the other on the display 730 with fullresolution. In other words, the size of the FOVs as defined in step 716is chosen such that each FOV can be displayed on the display 730 withfull resolution and without loss of information. The sequential orderwith which the FOVs of the sub-set are displayed may be determineddepending on a confidence value with which the selection criterion hasbeen fulfilled. For example, the first FOV of the sub-set that is shownon the display 730 is the one that has the highest confidence value ofmeeting the selection criterion used in the rule applied in step 720.

Alternatively or in addition lower resolution representations of theFOVs 722, 724 and 726 that form the sub-set in the example consideredhere may be displayed on the display 730 for selection via a graphicaluser interface of the image processing system 200. In response toselecting one of the FOVs of the sub-set, such as by a mouse click or apen or stylus entry action, the selected FOV is magnified to its fullresolution for detailed review by the observer.

In response to displaying a FOV, such as FOV 722 on display 730 asillustrated in FIG. 7, a signal may be received in step 731 via thegraphical user interface of the image processing system 200. Receipt ofthe signal in step 731 signals to the image processing system 200 thatthe FOV that is currently shown on display 730 with full resolution,i.e. FOV 722, contains a histopathological artefact or erroneousanalysis results

In response to receipt of the signal in step 731 an image portion 732that contains the FOV 722 for which the signal has been received isdisplayed on the display 730. This portion 732 may be the whole image700 or a portion of the image 700 that contains the FOV 722 and a regionthat surrounds the FOV 722. As depicted in FIG. 7 the FOV 722 is locatedwithin a histopathological artefact 734 in the example considered here.

In step 736 a selection of an image region to be excluded from theanalysis is received via the graphical user interface of the imageprocessing system 200. The selection of this image region may beperformed using an annotation tool of the graphical user interface. Forexample, the display 730 may be implemented as a pen display monitor. Bymeans of a pen or stylus 738 the observer may select the image regionthat contains the histopathological artefact 734 by tracing the boundaryof the histopathological artefact 734 by means of stylus 738. As aconsequence of this selection the image region that shows thehistopathological artefact 734 is excluded from the analysis, such thatany image processing result obtained in step 704 for that image regionwhich covers the histopathological artefact 734 is ignored and excludedfrom the result that is output in step 739. The resultant output image740 may show a heat map excluding the histopathological artefact 734.

The disclosed operations therefore mitigate tedious whole-slide QCprocedures that are either interactive or require a large number ofFOVs, while also avoiding incomplete QC where artefacts can be missed bychoosing FOVs that cover the extreme cases of detected target andnon-target objects. Moreover, besides medical applications such asanatomical or clinical pathology, prostrate/lung cancer diagnosis, etc.,the same methods may be performed to analysis other types of samplessuch as remote sensing of geologic or astronomical data, etc. Theoperations disclosed herein may be ported into a hardware graphicsprocessing unit (GPU), enabling a multi-threaded parallelimplementation.

Computers typically include known components, such as a processor, anoperating system, system memory, memory storage devices, input-outputcontrollers, input-output devices, and display devices. It will also beunderstood by those of ordinary skill in the relevant art that there aremany possible configurations and components of a computer and may alsoinclude cache memory, a data backup unit, and many other devices.Examples of input devices include a keyboard, cursor control devices(e.g., a mouse), a microphone, a scanner, and so forth. Examples ofoutput devices include a display device (e.g., a monitor or projector),speakers, a printer, a network card, and so forth. Display devices mayinclude display devices that provide visual information, thisinformation typically may be logically and/or physically organized as anarray of pixels. An interface controller may also be included that maycomprise any of a variety of known or future software programs forproviding input and output interfaces. For example, interfaces mayinclude what are generally referred to as “Graphical User Interfaces”(often referred to as GUI's) that provide one or more graphicalrepresentations to a user. Interfaces are typically enabled to acceptuser inputs using means of selection or input known to those of ordinaryskill in the related art. The interface may also be a touch screendevice. In the same or alternative embodiments, applications on acomputer may employ an interface that includes what are referred to as“command line interfaces” (often referred to as CLI's). CLI's typicallyprovide a text based interaction between an application and a user.Typically, command line interfaces present output and receive input aslines of text through display devices. For example, some implementationsmay include what are referred to as a “shell” such as Unix Shells knownto those of ordinary skill in the related art, or Microsoft WindowsPowershell that employs object-oriented type programming architecturessuch as the Microsoft .NET framework.

Those of ordinary skill in the related art will appreciate thatinterfaces may include one or more GUI's, CLI's or a combinationthereof. A processor may include a commercially available processor suchas a Celeron, Core, or Pentium processor made by Intel Corporation, aSPARC processor made by Sun Microsystems, an Athlon, Sempron, Phenom, orOpteron processor made by AMD Corporation, or it may be one of otherprocessors that are or will become available. Some embodiments of aprocessor may include what is referred to as multi-core processor and/orbe enabled to employ parallel processing technology in a single ormulti-core configuration. For example, a multi-core architecturetypically comprises two or more processor “execution cores”. In thepresent example, each execution core may perform as an independentprocessor that enables parallel execution of multiple threads. Inaddition, those of ordinary skill in the related will appreciate that aprocessor may be configured in what is generally referred to as 32 or 64bit architectures, or other architectural configurations now known orthat may be developed in the future.

A processor typically executes an operating system, which may be, forexample, a Windows type operating system from the Microsoft Corporation;the Mac OS X operating system from Apple Computer Corp.; a Unix orLinux-type operating system available from many vendors or what isreferred to as an open source; another or a future operating system; orsome combination thereof. An operating system interfaces with firmwareand hardware in a well-known manner, and facilitates the processor incoordinating and executing the functions of various computer programsthat may be written in a variety of programming languages. An operatingsystem, typically in cooperation with a processor, coordinates andexecutes functions of the other components of a computer. An operatingsystem also provides scheduling, input-output control, file and datamanagement, memory management, and communication control and relatedservices, all in accordance with known techniques.

System memory may include any of a variety of known or future memorystorage devices that can be used to store the desired information andthat can be accessed by a computer. Computer readable storage media mayinclude volatile and non-volatile, removable and non-removable mediaimplemented in any method or technology for storage of information suchas computer readable instructions, data structures, program modules, orother data. Examples include any commonly available random access memory(RAM), read-only memory (ROM), electronically erasable programmableread-only memory (EEPROM), digital versatile disks (DVD), magneticmedium, such as a resident hard disk or tape, an optical medium such asa read and write compact disc, or other memory storage device. Memorystorage devices may include any of a variety of known or future devices,including a compact disk drive, a tape drive, a removable hard diskdrive, USB or flash drive, or a diskette drive. Such types of memorystorage devices typically read from, and/or write to, a program storagemedium such as, respectively, a compact disk, magnetic tape, removablehard disk, USB or flash drive, or floppy diskette. Any of these programstorage media, or others now in use or that may later be developed, maybe considered a computer program product. As will be appreciated, theseprogram storage media typically store a computer software program and/ordata. Computer software programs, also called computer control logic,typically are stored in system memory and/or the program storage deviceused in conjunction with memory storage device. In some embodiments, acomputer program product is described comprising a computer usablemedium having control logic (computer software program, includingprogram code) stored therein. The control logic, when executed by aprocessor, causes the processor to perform functions described herein.In other embodiments, some functions are implemented primarily inhardware using, for example, a hardware state machine. Implementation ofthe hardware state machine so as to perform the functions describedherein will be apparent to those skilled in the relevant arts.Input-output controllers could include any of a variety of known devicesfor accepting and processing information from a user, whether a human ora machine, whether local or remote. Such devices include, for example,modem cards, wireless cards, network interface cards, sound cards, orother types of controllers for any of a variety of known input devices.Output controllers could include controllers for any of a variety ofknown display devices for presenting information to a user, whether ahuman or a machine, whether local or remote. In the presently describedembodiment, the functional elements of a computer communicate with eachother via a system bus. Some embodiments of a computer may communicatewith some functional elements using network or other types of remotecommunications. As will be evident to those skilled in the relevant art,an instrument control and/or a data processing application, ifimplemented in software, may be loaded into and executed from systemmemory and/or a memory storage device. All or portions of the instrumentcontrol and/or data processing applications may also reside in aread-only memory or similar device of the memory storage device, suchdevices not requiring that the instrument control and/or data processingapplications first be loaded through input-output controllers. It willbe understood by those skilled in the relevant art that the instrumentcontrol and/or data processing applications, or portions of it, may beloaded by a processor, in a known manner into system memory, or cachememory, or both, as advantageous for execution. Also, a computer mayinclude one or more library files, experiment data files, and aninternet client stored in system memory. For example, experiment datacould include data related to one or more experiments or assays, such asdetected signal values, or other values associated with one or moresequencing by synthesis (SBS) experiments or processes. Additionally, aninternet client may include an application enabled to access a remoteservice on another computer using a network and may for instancecomprise what are generally referred to as “Web Browsers”. In thepresent example, some commonly employed web browsers include MicrosoftInternet Explorer available from Microsoft Corporation, Mozilla Firefoxfrom the Mozilla Corporation, Safari from Apple Computer Corp., GoogleChrome from the Google Corporation, or other type of web browsercurrently known in the art or to be developed in the future. Also, inthe same or other embodiments an internet client may include, or couldbe an element of, specialized software applications enabled to accessremote information via a network such as a data processing applicationfor biological applications.

A network may include one or more of the many various types of networkswell known to those of ordinary skill in the art. For example, a networkmay include a local or wide area network that may employ what iscommonly referred to as a TCP/IP protocol suite to communicate. Anetwork may include a network comprising a worldwide system ofinterconnected computer networks that is commonly referred to as theinternet or could also include various intranet architectures. Those ofordinary skill in the related arts will also appreciate that some usersin networked environments may prefer to employ what are generallyreferred to as “firewalls” (also sometimes referred to as PacketFilters, or Border Protection Devices) to control information traffic toand from hardware and/or software systems. For example, firewalls maycomprise hardware or software elements or some combination thereof andare typically designed to enforce security policies put in place byusers, such as for instance network administrators, etc.

ADDITIONAL EMBODIMENTS Additional Embodiment 1

A method for analyzing a stained biopsy or surgical tissue sample beingimplemented by an image processing system (200), the method comprising:receiving (702) a whole-slide image (700) from a histopathologicaltissue slide of the stained tissue sample, analyzing (704) thewhole-slide image for detection of biological target objects byexecution of an image processing algorithm, applying (720) at least onerule to a plurality of field of views (FOVs) on the whole-slide imagefor identifying a set of the FOVs (722, 724, 726) for which a criterionis fulfilled that indicates a risk for the respective FOV containing ahistopathological artefact or an erroneous detection result caused bythe image processing algorithm, wherein the criterion relates to thedetection result obtained by execution of the image processingalgorithm, displaying (728) at least a sub-set of the set of FOVs forreview by an observer, in response to displaying a FOV of the sub-set,receiving (731) an entry signal from the observer that indicates whetherthe quality of an image portion of the displayed FOV is sufficient forthe analysis, outputting (739) a result of the detection of thebiological target objects depending on the entry signal.

Additional Embodiment 2

The method of embodiment 1, further comprising: receiving (736) aselection of the image region (734) to be excluded from the analysisfrom the observer via a graphical user interface of the image processingsystem, whereby the outputted result is expressive of the detectedbiological target objects excluding detection results obtained byexecution of the image processing algorithm from the excluded imageregion.

Additional Embodiment 3

The method of embodiment 2, the graphical user interface comprising anannotation tool, wherein the annotation tool is used for entry of theselection of the image region to be excluded from the analysis.

Additional Embodiment 4

The method of embodiment 2 or 3, wherein the image processing systemcomprises a data entry stylus (738), and wherein the selection of theimage region to be excluded from the analysis is performed by using thestylus.

Additional Embodiment 5

The method of embodiment 4, wherein the image processing systemcomprises a pen display monitor (730), and wherein the stylus is usedfor entry of a delimitation of the image region to be excluded from theanalysis via the pen display monitor by tracing the boundary of anartefact contained in the displayed FOV (722) of the sub-set.

Additional Embodiment 6

The method of any one of the preceding embodiments, wherein the size ofthe FOVs is selected for display with full resolution on a displaymonitor (730) of the image processing system.

Additional Embodiment 7

The method of embodiment 6, wherein neighboring FOVs are overlapping.

Additional Embodiment 8

The method of embodiment 7, wherein a FOV is defined by each pixelcontained in the whole-slide image.

Additional Embodiment 9

The method of any one of the preceding embodiments, wherein the numberof the FOVs contained in the sub-set is limited by a maximal number andwherein the set of the FOVs is sorted by means of a sorting criterion toprovide a sorted list and the maximum number of top ranking FOVs isselected from the sorted list to form the sub-set.

Additional Embodiment 10

The method of any one of the preceding embodiments, wherein the stainedbiopsy tissue sample is a whole mount sample, such as a prostratesample.

Additional Embodiment 11

An image processing system being configured for execution of a method inaccordance with any one of the preceding embodiments.

Additional Embodiment 12

A system (200) for quality control of automated whole-slide analysis,the system comprising: a processor; and a memory coupled to theprocessor, the memory to store computer-readable instructions that, whenexecuted by the processor, cause the processor to perform operationscomprising: applying a rule to a plurality of fields-of-view (FOVs) on awhole-slide image of a tissue specimen; and presenting a set of FOVsthat match the rule to an observer for a quality control analysis.

Additional Embodiment 13

The system of embodiment 12, wherein the operations further comprisedetermining a local density of one or more objects of interest in thewhole-slide image.

Additional Embodiment 14

The system of embodiment 13, wherein the local density is depicted by aheat map.

Additional Embodiment 15

The system of embodiment 14, wherein the rule is applied to the heatmap.

Additional Embodiment 16

The system of embodiment 15, wherein the one or more objects aredetermined to be of more than one type.

Additional Embodiment 17

The system of embodiment 16, wherein a plurality of heat maps aregenerated for a corresponding plurality of types of objects.

Additional Embodiment 18

The system of embodiment 17, wherein the local density is determinedbased on the rule.

Additional Embodiment 19

The system of embodiment 18, wherein the rule comprises a selection ofan FOV having a highest density of a target structure or biomarker.

Additional Embodiment 20

The system of embodiment 19, wherein the rule comprises a selection ofan FOV having a lowest density of a target structure or biomarker.

Additional Embodiment 21

The system of embodiment 20, wherein the rule comprises a selection ofan FOV having a density of a target structure or biomarker that isclosest to a target percentile of density within the whole-slide imageof said target structure or biomarker.

Additional Embodiment 22

The system of embodiment 21, wherein the rule is one among a pluralityof rules.

Additional Embodiment 23

The system of embodiment 22, wherein upon a determination that the setof FOVs that match the plurality of rules exceeds the number of rules,the operations further comprise selecting and purging a sub-set of FOVs.

Additional Embodiment 24

The system of embodiment 23, wherein purging the sub-set of FOVscomprises one or more of removing duplicate FOVs that match more thanone rule, or removing FOVs that overlap other FOVs in the set by athreshold overlap.

Additional Embodiment 25

The system of embodiment 24, wherein upon a determination that the setof FOVs that match the plurality of rules is smaller than the number ofrules, the operations further comprise optimizing the FOVs that matchthe rule.

Additional Embodiment 26

The system of embodiment 25, wherein the optimizing comprises selectingadditional FOVs from among the plurality of FOVs that have an overalllowest distance to an FOV that best matches the rule, in addition to theFOV that best matches the rule.

Additional Embodiment 27

The system of embodiment 26, wherein the overall lowest distance isdetermined by sorting the FOVs in a list or table.

Additional Embodiment 28

The system of embodiment 12, wherein each of the plurality of FOVs has aunique identifier.

Additional Embodiment 29

The system of embodiment 12, wherein the operations further comprisegenerating the plurality of FOVs based on one or more of a resolution ofthe image or a resolution of a display device.

Additional Embodiment 30

A computer-implemented method for quality control of an automatedwhole-slide analysis, the method comprising: determining a local densityof objects of one or more types in a whole-slide image; selecting aplurality of rules based on the local density; and selecting a set offields-of-view (FOVs) that fulfill the plurality of rules; wherein oneof the rules sets a maximum number of FOVs within the set of FOVs.

Additional Embodiment 31

A tangible non-transitory computer-readable medium to storecomputer-readable code that is executed by a processor to performoperations comprising: segmenting a whole-slide image into a pluralityof fields-of-view (FOVs); and selecting a set of FOVs that match one ormore rules for the selection of FOVs; wherein the rules are applied toheat maps representing a local density of one or more objects ofinterest in the whole-slide image; and wherein the set of FOVs arepresented for quality-control of an image analysis algorithm applied tothe whole-slide image.

The foregoing disclosure of the exemplary embodiments of the presentsubject disclosure has been presented for purposes of illustration anddescription. It is not intended to be exhaustive or to limit the subjectdisclosure to the precise forms disclosed. Many variations andmodifications of the embodiments described herein will be apparent toone of ordinary skill in the art in light of the above disclosure. Thescope of the subject disclosure is to be defined only by the claimsappended hereto, and by their equivalents.

Further, in describing representative embodiments of the present subjectdisclosure, the specification may have presented the method and/orprocess of the present subject disclosure as a particular sequence ofsteps. However, to the extent that the method or process does not relyon the particular order of steps set forth herein, the method or processshould not be limited to the particular sequence of steps described. Asone of ordinary skill in the art would appreciate, other sequences ofsteps may be possible. Therefore, the particular order of the steps setforth in the specification should not be construed as limitations on theclaims. In addition, the claims directed to the method and/or process ofthe present subject disclosure should not be limited to the performanceof their steps in the order written, and one skilled in the art canreadily appreciate that the sequences may be varied and still remainwithin the spirit and scope of the present subject disclosure.

The invention claimed is:
 1. A tangible non-transitory computer-readablemedium to store computer-readable code that is executed by a processorto perform operations comprising: (i) obtaining a whole-slide image of abiological sample stained for the presence of one or more biomarkers;(ii) determining one or more local densities of one or more objects ofinterest in the obtained whole-slide image; (iii) segmenting theobtained whole-slide image into a plurality of fields-of-view (FOVs);(iv) identifying one or more FOVs from the plurality of FOVs that meetone or more predetermined density rules based on the determined one ormore local densities of the one or more objects of interest; and (v)generating one or more heat maps, wherein each heat map represents oneor more of the determined local densities, wherein the one or morepredetermined density rules are applied to one or more generated heatmaps.
 2. The tangible non-transitory computer-readable medium of claim1, wherein the one or more objects of interest are cells positive for afirst biomarker.
 3. The tangible non-transitory computer-readable mediumof claim 1, wherein the one or more objects of interest are cellspositive for a second biomarker.
 4. The tangible non-transitorycomputer-readable medium of claim 1, wherein the one or more objects ofinterest are cells positive for both a first biomarker and a secondbiomarker.
 5. The tangible non-transitory computer-readable medium ofclaim 1, wherein the one or more objects of interest are “objects of nointerest.”
 6. The tangible non-transitory computer-readable medium ofclaim 1, wherein the one or more predetermined rules comprise aselection of an FOV from the plurality of FOVs having a highest densityof a target structure or biomarker.
 7. The tangible non-transitorycomputer-readable medium of claim 1, wherein the one or morepredetermined rules comprise a selection of an FOV from the plurality ofFOVs having a lowest density of a target structure or biomarker.
 8. Thetangible non-transitory computer-readable medium of claim 1, wherein theone or more predetermined rules comprise a selection of an FOV from theplurality of FOVs having a biomarker density that meets a predeterminedthreshold value.
 9. The tangible non-transitory computer-readable mediumof claim 8, wherein the predetermined threshold value is for a singlebiomarker.
 10. The tangible non-transitory computer-readable medium ofclaim 8, wherein the predetermined threshold value is a ratio of a firstbiomarker density to a second biomarker density.
 11. The tangiblenon-transitory computer-readable medium of claim 1, further comprisingoutputting to a display device at least one FOV from the set of selectedFOVs that match the predetermined rules.
 12. A tangible non-transitorycomputer-readable medium to store computer-readable code that isexecuted by a processor to perform operations comprising: (i) obtaininga whole-slide image of a biological sample stained for the presence ofone or more biomarkers; (ii) determining one or more local densities ofone or more objects of interest in the obtained whole-slide image; (iii)segmenting the obtained whole-slide image into a plurality offields-of-view (FOVs); and (iv) identifying one or more FOVs from theplurality of FOVs that meet one or more predetermined density rulesbased on the determined one or more local densities of the one or moreobjects of interest, wherein the one or more predetermined density rulescomprise a selection of an FOV from the plurality of FOVs having alowest density of a target structure or biomarker.
 13. The tangiblenon-transitory computer-readable medium of claim 12, further comprisingoutputting to a display device at least one FOV from the set of selectedFOVs that match the predetermined rules.
 14. A tangible non-transitorycomputer-readable medium to store computer-readable code that isexecuted by a processor to perform operations comprising: (i) obtaininga whole-slide image of a biological sample stained for the presence ofone or more biomarkers; (ii) determining one or more local densities ofone or more objects of interest in the obtained whole-slide image; (iii)segmenting the obtained whole-slide image into a plurality offields-of-view (FOVs); and (iv) identifying one or more FOVs from theplurality of FOVs that meet one or more predetermined density rulesbased on the determined one or more local densities of the one or moreobjects of interest, wherein the one or more predetermined density rulescomprise a selection of an FOV from the plurality of FOVs having abiomarker density that meets a predetermined threshold value, andwherein the predetermined threshold value is a ratio of a firstbiomarker density to a second biomarker density.